Empirical Evaluation of Software

Why has there been so little progress in the quantitative evaluation of those quality attributes concerned with software design? This paper seeks to provide an answer by arguing that progress is dependent on the adoption of a programme of cumulative empirical research. Such a programme consists of original empirical research, independent replication to investigate and verify original results, and, critically, recipe-improvement replication in an attempt to produce more generalized results. Furthermore, a programme of cumulative empirical research is dependent upon quality attributes being communally deened and soundly related to appropriate measures. Empirical research is necessary to demonstrate that quality attributes, and their quan-tiication, are well-founded. Replication is necessary to investigate and connrm original results. Thereafter, recipe-improvement addresses the signiicant problems of subject, environment and domain variability associated with the empirical study of software quality attributes. Recipe-improvement research is essential if results are to be progressed from the speciic to the general. To support this argument this paper critically reviews the empirical research investigating the quality attribute of module coupling. Module coupling has been advocated for at least two decades. As an attribute of software quality, it is theoretically convincing, generally accepted, and widely taught. Furthermore, it is one of the few quality attributes that have been subject to a number of well publicised empirical studies. The empirical evidence accumulating from these studies, however, is shown to be insubstantial. The key empirical studies to date have diiered in their deenition of module coupling, the eeects sought, the operational measures used, and the results found. It is concluded that this series of single shot studies must be replaced by a programme of cumulative research.